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ABSTRACT 



This paper describes a five-phase, 20-week, computer 
supported reading comprehension instruction process, which begins with access 
to powerful supports and direct teacher-mediated instruction. The process 
involves five phases: (1) fully supported reading and strategy instruction; 

(2) strategy practice in a fully supported reading environment with teacher 
review of generated comprehension strategy questions; (3) independent 
strategy practice in a partially supported environment with teacher review of 
generated comprehension strategy questions; (4) strategy practice in a 
partially supported environment with peer review of generated comprehension 
strategy questions; and (5) generalization to a paper-based text in a 
partially supported environment with teacher and group review of generated 
comprehension strategy questions. Several elements are common to all phases: 
all instruction occurs within the context of a resource room or separate 
classroom for students with learning disabilities; each phase is 
approximately 4 weeks duration, with at least 10 hours of instructional time; 
during each phase, new text is used; and students working in pairs take turns 
using the speech-to-text capability to dictate their responses during 
alternating instructional sessions. Project development, implementation, and 
evaluation are described. (SM) 
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PROJECT ABSTRACT 



Engaging the Text: Reciprocal Teaching and Questioning Strategies in a Scaffolded 

Learning Environment 

1. In the area of reading comprehension, students with learning disabilities and their 
teachers have at least one thing in common - while thej both learned effective 
strategies, they may be unable to bring them into play. For students, the process of 
decoding words requires so much effort that they are often unable to approach text 
strategically to construct its meaning, in spite of having been taught helpful strategies. 
Likewise, many teachers know about labor intensive teaching strategies which have 
been shown to be successful in helping students, but they simply do not have the time 
or resources to implement them consistently. 

2. A common solution in the face of such limitations is to multiply limited capacity by 
with better tools. In the proposed project we seek to investigate whether there are 
better classroom tools available to support students with learning disabilities in 
becoming strategic readers. 

3. Our research plan is a mirror of what we intend for our teachers and students. We will 
use existing strategies for teaching students with learning disabilities, but we will 
apply those strategies in the context of a more supportive technology than those 
usually found in classrooms. We propose to combine promising strategies with 
supportive technologies and then to asses whether this combination creates a better 
environment for both the teaching and learning. 

4. The teaching strategies we will use are adopted from the Reciprocal Teaching Method 
which has been developed and researched over the last two decades. We will 
integrate these methods for developing active reading strategies within traditional 
curricular content that has been significantly enhanced through text-to-speech and 
speech-to-text technologies. In this digital form there will be many supports for the 
apprentice reader ^ in both decoding and strategy development. These new highly 
supportive curricular materials will provide scaffolding for students during instruction 
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and practice. As the student develops competency in active reading, electronic and 
pedagogical supports will be gradually withdrawn. 

5. CAST will assess the success of this method in helping learning disabled middle 
school students develop comprehension skills and achieve success that is evident 
across reading contexts through a systematic, controlled evaluation of the method. 
Results will be disseminated through avenues that will affect both the kinds of 
curricula that are being developed - at a time when publishers are seeking digital 
solutions to demands for inclusion - and the strategies for using the new kinds of 
curricular technologies that are arriving in every classroom in America. 
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The Intervention: 



We have devised a five phase, twenty week process of computer supported 
reading comprehension instruction, beginning with access to powerful supports and direct 
teacher- mediated instruction. The process proceeds through three support fading and 
practice phases to a final phase of generalization in an ordinary, unsupported, text-on- 
paper environment. ' 

Several elements are common to all phases of our proposed instructional 
technique: a) All instruction will take place within the context of a resource room or a 
substantially separate classroom for students with learning disabilities, b) Eachf)hase is 
of approximately four weeks duration, encompassing a minimum of ten hours of 
instructional time, c) During each phase, new text will be used, d) Students working in 
pairs will take turns using the speech-to-text capability to dictate their responses during 
alternating instructional sessions. 

Bhase One: Fully supported reading and strategy instruction: . - 

Students, in groups of up to four, will work directly with a teacher and the 

computer. Teachers will instruct the students in Reciprocal Teaching, including the 
importance of monitoring their own understanding of text as they read, and in the 
generation of appropriate questions with which to do this. Students will also be taught to 
use CAST’s eReader™ text to speech software and Naturally Speaking™ or other 
appropriate dictation (speech-to-text) program both separately and in combination. 

The eReader will read digitized versions of their own Language Arts books 
(primarily novels), prepared with embedded questions of the sorts that students are 
learning to generate as they read, into audible spoken words. This use of copyrighted 
material is permitted under the doctrine of fair use. This will: a) support students’ 
decoding skills, allowing them to concentrate on comprehension, b) expose them to 
appropriate strategies in the context of reading materials, and c) familiarize them with 
comprehension monitoring questions in context, modeling the generation process that 
they will undertake independently in subsequent phases. 
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Phase Two: Strategy practice in a fully supported reading environment with 

teach er review of generated comprehension strategy questions: "■ 

In groups of two, students will use the text-to-speech capacity of CAST’s 

eReader'^'^ to “read” chapter length passages from their own Language Arts books. At 

intervals in the prepared texts, the computer’s voice will prompt students to generate, ask 

themselves, and answer comprehension questions of the sort they were taught to generate 

and use during the first phase. One of the pair will “write’,’ the questions into the passage 

at the prompts using the Naturally Speaking™ speech-to-text system. 

When the passage has been read and the questions generated, students will save 

their work electronically and print a copy out for review by their teachers and the 

experimenters. Teachers will give students feedback on the appropriateness of the 

generated questions, and provide further instruction as needed. 

Phase Three: Independent strategy practice in a partially supported environment with 
teache frei^w of generated comprehension strategy questions: ~ ^ ^ : ' " r 

In groups of two, students will use the text-to-speech capacity of CAST’s 

eReader™ to “read” chapter length passages (but nol prompts to ask themselves 

questions) from their own Language Arts books. At intervals in the prepared texts 

embedded icons that “speak” directions when clicked with the mouse will prompt 

students to generate, ask themselves, and answer comprehension questions. One of the 

students will “write” the questions into the passage using the Naturally Speaking™ 



speech-to-text system. 

When the passage has been read and the questions generated, students will save 
their work electronically and print a copy out for review by their teachers and the 
experimenters. Teachers will provide feedback on the appropriateness of the generated 



questions. 



Phase Four: Strategy practice in a partially supported environment with peer review of 
generated comprehension strategy questions: ^ 

In groups of two, students will read chapter length passages to themselves. These 



passages will be presented on the computer’s screen, but the text-to-speech capacity will 
not he available . Students will generate, ask themselves, and answer comprehension 
strategy questions as they have before, but they will not be prompted when to generate 
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them, with one student “writing" the questions into the passage using the Naturally 

Speaking^”^ or other speech recognition software. When the passage has been read and 

the questions generated, students will save their work electronically and a copy of the 

computer file will be given to the teacher. This file will later be provided to a pair of 

students who are to read the same passage. The second pair will use the student-prepared 

comprehension strategy questions and, in a brief meeting with the teacher and the 

students who generated the questions, share their impressions of the usefulness of the 

questions and make suggestions for improvement where warranted. 

Phase Five: Generalization to paper based text in a partially supported environment 

with teacher and group review of generated comprehension strategy questions: 

In groups of two, students will again read. Their passages will be presented in standard 

written form, but photocopied copies will be used so that students can mark on the text. 

At intervals in the prepared texts, students will generate, ask themselves, and answer 

comprehension strategy questions. One will “write” the questions into a computer file 

using the Naturally Speaking™ speech-to-text system.' When the passage has been read 

and the questions generated, students will save their questions electronically and print a 

copy out for review by their teachers and the experimenters. 

Research Design 

Year 1: Development of the instmctional technique through f ormative evaluation: 
(1998-1999) During the first year of the study, the focus will be on developing the 
instructional technique. To this end, a formative evaluation consisting of four 
overlapping development cycles will be conducted, and the methods used to collect 
information during this phase will be primarily those of qualitative inquiry (Lawson, 

1974; Flagg, 1990). 

Research Question: What elements should be built into an instructional technique 
that integrates Reciprocal Teaching plus technology-based supports for practice to help 
students with learning disabilities take an active role in their own reading comprehension 
by engaging in self-questioning and other validated comprehension strategies? 
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This phase consists of four overlapping cycles (see table 1) In the first cycle, we 
will develop and test teacher procedures for reading comprehension and questioning 
strategies instruction, and test usability of existing elements of the technology (CAST’s 
eReader™ and Naturally Speaking™). In cycle two, we will focus on different ways of 
incorporating the four elements of reciprocal teaching: a) questioning, b) summarization, 
c) prediction and d) seeking clarification on unclear points, into formatted text to be 
“read” by the computer. In cycle three, we will examine the frequency 'and placement of 
prompts in text. In cycle 4, we will combine what we have learned in the previous cycles 
and apply it to a paired practice situation. 

Table 1: Timeline* 



PHASE 1 (Year 1) 

Systematic formative evaluation to develop the new instructional technique 





Exploration 


Development 


Trial 


Refinement 


Cycle 1: 


2 week 


2 week 


1 month 


2 week 


exploration 


development 


trial 


refinement 
( Cycle 2 begins) 


Cycle 2: 


2 week 


2 week 


1 month 


2 week 


exploration 


development 


trial 


refinement 
( Cycle 3 begins) 


Cycle 3: 


2 week 


2 week 


1 month 


2 week 


exploration 


development 


trial 


refinement 
( Cycle 4 begins) 


Cycle 4: 


2 week 


2 week 


1 month 


2 week 


exploration 


development 


trial 


refinement 



PHASE 2 (Year 2) 

Twenty week pilot study of the instructional technique 

Pre testily Trial Post- Final 

testing adjustment 

PHASE 3 (Year 3) 

Twenty week field testing and validation of the instructional technique 

Pretesting Intervention Post- 

testing 

CASE STUDY (Encompasses Phases 2 & 3, Year 2 - Year 3) 

' Please note that while conducting the evaluation, changes in the timeline may occur because of school vacations. 
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Within each cycle, there are four activities (see Table 1): a) Exploration is two- 
weeks of intense work with one group of five students, exploring target elements in the 
instructional technique, b) Development is one-month of materials and procedure 
development based on data gathered during Exploration, c) Trial is a one-month 
assessment of a more refined implementation of the instructional technique, and d) a one- 
month Refinement period based on data gathered during the Trial cycle. 

Year 2: Pilot testing of the instructional technique : (1999-2000) During the 
second year of the study, the focus will be on testing the instructional technique during a 
twenty week pilot study; qualitative and quantitative techniques will be used. While not 
a formal evaluation, it will be in most respects summative. Intensive case studies of four 
students using the instructional technique will be initiated, and will continue through the 
third year. These will help us gauge the degree of generalization of active, strategic 
reading and resulting improvement to comprehension of classroom work. 

Research Question: Does the newly developed instructional technique show 
promise to help students learn to read more strategically and so improve their reading 
comprehension? 

During this phase, approximately 36 students will participate. At least 18 students 
will use the instructional technique and an equal number will serve as a control group, 
using standard written materials and ordinary instruction. Pre-tests measuring 
standardized reading comprehension and attributions/self-concepts will be administered 
to all students. Students in the intervention group will begin the 5 phase, twenty-week 
process of instruction plus computer supported reading comprehension strategy practice. 
Students in the control group will use standard text-based materials and receive teachers’ 
customary instruction. Students will be observed using the approach twice weekly. 
Student and teacher feedback will be used to make final adjustments to the instructional 
technique. At the end of the twenty week process, students and teachers will be 
interviewed. 
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For both groups, work samples will be collected regularly and both experimenter 
designed assessments of reading comprehension and post-tests (the same measures used 
in pre-testing) will be administered. 

Case Studies Investigate generalization of the instructional technique, case 
studies of four students chosen at random from one of the classrooms using the Year 2 
intervention will be conducted during the second and third years of the project. These 
students will meet our stringent year-three criteria for identification as having a learning 
disability. During the second year of the study, the students will use the instructional 
technique, and will be monitored to gather information about their work and progress. 
Case study activities will include: Bimonthly observations of the students using -the 
instructional technique, bimonthly observations of the students during reading periods, 
copies of student work in reading comprehension, pre and post test data in reading 
comprehension, questioning strategies and attributions/self-concepts around academics, 
interviews with students and teachers and collection of background information (e.g. lEP, 
etc.). In the third year of the study (the second year of the case study), these students will 
not be included in any of the three experimental groups, nor receive any instruction 
through this project. However, the standardized reading comprehension measures, 
observations and other data collection procedures will continue. Final case studies will 
be comprised of two years of data on four individuals. 

Year 3: Evaluation of the instructional technique : (2000-2001) During the third 
year of the proposed study, the focus will be on measuring the effectiveness of the 
technology-enhanced intervention through a formal summative evaluation, employing a 
control group and random selection and assignment of students to control or intervention 
groups. Several schools will participate in this phase of study, yielding a sample of at 
least 80 middle school students with learning disabilities. Qualitative and quantitative 
techniques, including standardized measures of reading comprehension, will be used to 
collect efficacy information. 

Research Question: Do learning disabled middle school students develop active 
strategic reading skills and reading comprehension ability in expository texts more 
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rapidly when taught using: a) the Reciprocal Teaching method and standard texts, or b) 
the newly developed technology-supported technique and electronic texts? 

Procedures: The sample (n=80) will be divided at random into two groups: 
“traditional” Reciprocal Teaching, and the technology-enhanced Reciprocal Teaching 
group. The “traditional” Reciprocal Teaching group will use standard written materials 
and will receive instruction using Reciprocal Teaching. The second group use the final 
form of our new instructional technique, including digitized text and technology supports. 
Every effort will be made to hold direct instruction and practice time constant across the 
three groups: 2 1/2 hours per week for 20 weeks. 

Pre-tests, post-tests, and interviews will be administered and student work 
collected. 

(2) Sample 

Students identified as having learning disabilities according to the Individuals 
with Disabilities Act Amendments of 1997, PL 105-17, criteria and determined to be at 
least two years behind age expectancy in reading comprehension skills will participate in 
the study. We will consult our advisory board members to ensure that we remain current 
with any changes and refinements within the definition and assessment of students with 
learning disabilities which might evolve during the project. 

The following quantitative background information will be gathered on the 
sample: Computer skills, age, gender, socioeconomic status, ethnicity, and supportive 
services provided by the school. Informed consent will be obtained from the parents of 
all students selected to participate in this project evaluation, as well as from the students 
themselves, and the privacy of ^e students will be respected throughout. Table 2 below 
outlines the samples to be used in each year of the study. 

Table 2: The samples in each year of the proposed study 



Minimum 

sample 

sizes 


Selection 

strategies 


Number of 
schools 


Number of 
classrooms 


Groups 


YEAR 1 20 


Not randomly 
selected 


1 


3 


4 groups of 5 
students 
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YEAR 2 


36 


Not randomly 
selected 


2 


At least 3 


2 groups of 18 
students 
(matched by 
gender & age) 


YEARS 


80 


Randomly 
selected & 
assigned 


several 


several 


40 students in 
each of the 2 
experimental 
group 

(matched by 
gender, age, & 
comprehension 



) 



Year 1: 

Four small samples (n = 5) of students with learning disabilities will be drawn for 
in this phase of the study. The samples will not be randomly selected nor matched by 
gender or other characteristics as this is not necessary during a formative evaluation. 

Year 2: 



A randomly selected sample of at least 36 students with learning disabilities, 
taken from at 2 middle schools, will be drawn for this year of the study. Half of the 
sample will be randomly assigned to use the new instructional technique and the other 
half will form a control group whose instruction will be “traditional” Reciprocal 
Teaching (refer to Table 2). If possible, students will be selected from 1 classroom in 
each school to minimize teacher effects, and groups will be matched by gender and age. 
Otherwise, class membership will be addressed as background data. 




Table 3: Experimental groups in the second year of the study 





Materials 


Focus of comprehension instruction 


Control Group 


Standard written 
materials 


Whatever teachers currently provide 


Technology Enhanced 
Reciprocal Teaching 
Group 


Standard written 
materials plus digitized 
and enhanced version of 
materials presented in 
CAST’s eReader. 


Reciprocal Teaching instruction, paired 
with strategic reading practice on 
computer. 
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Year 3: 



A randomly selected sample of at least 80 students with learning disabilities from 
several middle schools will be drawn. Three groups are used in this phase of study 
matched by gender, age and reading comprehension levels. Table 3 below outlines these. 



Table 3: Experimental groups in the third year of the study (n = at least 40 per group) 





Materials 


Focus of comprehension 
instruction 


Reciprocal Teaching Group 
(Control) 


Standard written materials 


Reciprocal Teaching instruction 
& practice 


Technology Enhanced 


Standard written materials 


Reciprocal Teaching 


Reciprocal Teaching Group 


plus digitized and enhanced 
materials presented on 
computer. 


instruction, paired strategic 
reading practice on computer. 



(3) Instrumentation 

Qualitative data collection methods will include: student and classroom 
observations, student and teacher interviews, user feedback, copies of student work, and 
assessment of questioning strategies. Protocols for the student and classroom 
observations will be developed, in concert with teachers and project participants, and will 
consist of both systematic and anecdotal avenues for recording students’ behaviors during 
instruction and practice. Protocols for the student interviews will be developed, 
consisting of questions about their use of the instructional technique, their use of 
comprehension strategies and their opinions about reading and about themselves as 
readers. Teacher interviews will consist of questions about their use and effectiveness of 
instructional technique, attitudes towards the questioning strategies, and feedback about 
student performance on reading comprehension tasks. Additional data on use of 
questioning strategies will come from student reports of what strategies they use while 
reading (elicited at intervals). We will observe the behavior of student pairs as they 
develop questions for specific passages and explain them to a peer in order to gain insight 
into the strategies that the student “teacher” knows and uses. 




Engaging the Text 
Pisha & Brady, CAST 



13 



Page 12 



Quantitative data collection methods will include: a standardized measure of 
reading comprehension, experimenter-designed measures of reading comprehension, an 
experimenter-designed measure of questioning strategies, and two measures that assess 
students’ attributions of academic success or failure and self-concepts as learners. The 
Stanford 9 Diagnostic Reading Test was selected as the standardized measure of reading 
comprehension because it is psychometrically robust, highly regarded, and it is 
administered annually to all middle school students in Boston Public Schools. The 
experimenter designed measures of reading comprehension and reading strategy use will 
be developed in concert with participating teachers and advisory board member Scott 
Paris during the course of the project. Types of comprehension questions will include 
both literal/factual and interpretive/inferential. Measures of strategy use will include 
forms of both direct student reports and think-alouds. Portions of the Harter Perceived 
Competence Scales (Harter, 1982) will be used to assess the students’ attributions and 
self-concepts. 

(41 Data Analysis Proceedures 

Year I data are collected to inform the development process, and so analysis will 
be systematic but informal, resulting in written recommendations that will guide the 
development and refinement of the instructional technique. The quantitative data 
collected during year 2 and year 3 wall be analyzed to compare results of the 
experimental and control groups, as well as wdthin groups (technology enhanced 
Reciprocal Teaching group, “traditional” Reciprocal Teaching control group), in reading 
comprehension, questioning strategies and attributions/self-concept. For each year’s 
data, univariate analyses will b^performed, and three gain scores wall be calculated for 
each student (reading comprehension, questioning strategies and attributions/self- 
concept) by subtracting pre-test scores from post-test scores. Then, three regression 
models wall be fit (each using one of the gain scores as the outcome variable, 
experimental group as a predictor) to further examine differences between and within 
experimental and control groups. Independent variables representing background 
information gathered on students wall be created and included in models both individually 
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and in groups to determine which, if any, are appropriate for use as control variables. 
Similarly, appropriate interaction variables will be created, and their predictive value 
evaluated. Residual analyses will be conducted to insure that no unusual individual cases 
have unduly influenced the analysis. 

The qualitative information will be analyzed in different ways, depending upon 
the purpose for which it was gathered. (Miles & Huberman, 1994) 

Case study data are collected to allow us to construct a detailed picture of the 
processes undertaken by four students as they use the intervention technique. The data 
for each student will be aggregated and a coding scheme will be developed to capture 
dimensions of interest. Using Nud»ist, a computer program designed for the task (Sage, 
1996), the codes will be assigned to the data which allow simultaneous examination of all 
data pertaining to any dimension of interest. This will allow efficient construction of 4 
narrative case studies (Miles & Huberman, 1994) that will inform discussions of the 
approach’s efficacy, generalization to new contexts, and potentially useful future 
refinements. 
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